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ABSTRACT 

A model for evaluation planning of post-literacy 
programs involves listing possible evaluation needs, administration 
of evaluation, and dissemination of evaluation results for 
utilization by all potential users. Questions that should b^pursued 
in evaluating post-literacy can be divided into these categories: (1) 
questions at the interface regarding literacy programs, literates, 
and aspirations of illiterate and new literate; (2) questions on 

^literacy retention; (3) questions on second chance education 
programs; (4) questions on the systemic integration theme (social 
integration of new literates); arid (5) questions on the socialization 
of individuals by post-literacy programs. Those working with 
post-literacy programs should choose questions from these lists to 
meet their priority needs. The tasks in the administration of 
evaluation include the development of an evaluation system; linkage 
of this system with the program system as well as with outside 
support institutions; recruitment and training of evaluators; 
resource generation and allocation; and provision of logistical 

• support, quality control, and time budgeting of the enterprise. The 
information collected must then be synthesized. From the syntheses of 
the evaluative information can be drawn practical operational 
conclusions that should be disseminated throughout the system among 
evaluators and program specialists, (YLB) 
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EVALUATION PLANNING OF POST-LlTERACY PROGRAMS 
By H.S. Bhola 

Evaluation planning is the process of generating an evaluation plan 
(or an evaluation agenda) within the context of a particular educational 
or development program, in a particular historical and social setting. 
Evaluation planning is not a mere technical process; it is a socio- 
technical process. Evaluation plans and agendas prepared by professional 
evaluators must be put through the political process of negotiation with 
policy makers, on the one hand, and with the beneficiaries of programs, 
on the other l.and. , " 

This paper can deal only with half the process of evaluation 
planning ~ the technical half. The other half of the process, that of 
political negotiation, can not be condjacted on paper. Indeed, it must 
be handled in real time and space, by real people, projecting their 
different values and defending thein diverse interests. That clearly 
we must leave to the real stakeholders within the program system. In 
accomplishing the technical part of evaluation planning, we will delineate 
some possible evaluation needs in the area of post-literacy. Based on 
such a list, evaluators of post-literacy programs in India, at this point 
in time, will have to develop their own priorities. As part of this 
evaluation planning exercise, we will also deal with two related planning 
concerns, those of the adminis tration of evaluation and the dissemination 

/ 

of evaluation results for utilization by all potential users. 



What is Post-Literacy? 

The word "evaluand" has been a useful addition to the vocabulary of 
evaluation. Evaluand is what is to be evaluated, the product, process, 
role, project, program or campaign that is the object of evaluation, the 
focus of the evaluator's concern. In our case, all that is subsumed 
under the phrase post-literacy is the evaluand. 

The very first question to be asked for evaluation planning (or for 
that matter for the administration or implementation of evaluation) is: 
What is the evaluand? We must, thus, inevitably face the question: What 
is post-literacy? 

Post-Literacy: The Time Frame 
and'^^the Program Frame 

Post-literacy as a concept i^s a mix of the t emporal (the time 
dimension) and of the programmatic (the program dimension) . Post- 
literacy in its program dimension consists of programs and projects that 
are built upon the assumption of literacy (or at least without the 
assumption of illiteracy) among those sought to be served by post- 
literacy programs. Thus, illiteracy is not the criterion used for the 
choice of modes or media of instruction in post-literacy programs, even 
though these programs may often serve both the literate and the illiterate. 
Paradoxically, post-literacy programs are not always for the already 
literate. 

In their temporal dimension, postrliteracy programs should follow 
literacy programs in time. This is typically so when seen from the 
perspective of the participants of one particular .literacy program. But 
time is relative from the perspective of planners of post-literacy programs. 
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Planning of literacy and post-literacy programs can and indeed must be 
handled concurrently and conjointly • The implementation of literacy 
and post-literacy programs will also be often concurrent because while 
some client groups will need initial literacy instruction, some others 
will need post-literacy programs, having acquired their literacy skills 
in earlier adult literacy programs, or within the context of the school. 

Clients of Post-Literacy Programs 

Unfortunately, the word post-literacy has often encouraged a fixation 
on the needs of adults coming out of literacy classes organized for them 
in out-of-school settings. Thus, only those coming out of adult literacy 
classes have been seen to be the clients of post-literacy programs. This 
is a narrow conception to have for the planner of post-literacy programs. 

Post-literacy programs should be planned to serve all those who are 
literate irrespective of the setting in which literacy was first acquired 
by the now literate. This means that clients of post-literacy programs 
^ will include adult men and women coming out of literacy classes especially 
organized for them; school leavers and dropouts from school, with various 
levels of literacy acquisition; and any others who may have acquired 
initial literacy at home, at work, in a religious establishment or in the 
army. Some may be self-taught. 

Four General Objectives of 
Post-Literacy Programs 

Four general objectives of post-literacy programs can be identified: 



^Bhola H.S* (with Joginder K. Bhola). 1980. Program and Curriculum 
Development ' in the Post-Liter acy Stages . (A Workshop Manual). Bonn, FRG: 
German Foundation for International Development, p. 16. 
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(i^) Literacy Retention ; This involves reinforcement and strengthening 
of literacy skills of new literates through use, making relapse into 
illiteracy unlikely. Sometimes there may be a prior task, that of remedi- 
ation, when the level of literacy acquisition among participants is found 
not to be of a sufficient level. 

(ii.) Second Chance Formal Education ; Tne objectives of post-literacy 
may often involve providing a second chance for formal primary or 
sezondary education to those who had missed the chance of going to school 
when young, or are unable to go Lo school now because there is no school 
ro go to in the community. 

(iii.) Systemic Integration ; The now literate are taught additional 
literacy skills and functional knowledge with the objective of helping 
chem to integrate within the social, economic and political institutions 
.And structures of the society to the mutual advantage of both the individ- 
ual and the social system. ^ 

(iv.) Socialization for an Ideal Society ; The now literate are prepared 
for creating, sustaining and participation in an ideal community and for 
contributions to a new world order. 

P ost-Literacy and Some Related Terms 

In discussions of out-of-school education in general, one comes 
across terms such as post-literacy and follow-up programs, nonformal 
education, continuing education, lifelong education and some other terms 
as well. It will be useful to understand the areas of overlap as well 
as some of the distinctions among these various terms. 

Post-literacy programs and nonformal education are almost equivalent 
as concepts, except that nonformal education does not raise the literacy/ 
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illiteracy question; and seeks to encompass the needs and interests of 
both the literate ard the illiterate learner groups. Pos^-literacy 
programs, however, while not actively and systematically excluding the 
illiterate, are planned on the assumption of literacy among its clients; 
and typically reinforce and build upon the literacy skills of their 
client groups* Follow-up programs are often seen as those focussed on 
reading materials, and reading circles and groups. 

The term continuing education is typically applied to education 
offered by schools or universities as part of an extension effort. On 
the other hand, the term continuing education is used to cover alternatives 
to the regular stream of formal education — formal education offered by 
systems competing with the formal system such as night schools, spare-time 
schools or correspondence education. Both post-literacy programs and 
nonformal education thus include continuing education. 

Lifelong education is co-existential with life. It is education 
from the cratlle to the grave; and it includes formal education, nonformal 
education, post-literacy education, continuing education, informal 
education as well as the learning possibilities of institutions of 
production, cooperation, welfare, governance and justice. 

Post-Literacy and Social Action 

Post-literacy programs at their best must connect education with 
social action. Post-literacy programs must provide learnings that can 
be often directly and immediately put to use in life and at work, yec 
organizers of post-literacy programs must retain a clear focus on the 
e^'ucational component of their programs and must not drift into service 
functions. They should also be careful so as not to encroach upon the 
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function^ of other development extension agents in agriculture, health, 
nutrition, family planning and the like. 

Program Planning and Implementation Versus 
Evaluation Planning and Implementation 

Ideally, evaluation planning should be part of program planning. 

Evaluation must be built into the programs so that program planners can 

know the terrain of their work, can conduct effective social diagnoses, 

can design appropriate interventions, and can collect systematic feedback 

for studying the impact of their programs on the lives of individuals and 

communities • 

Some implementation of evaluation must indeed precede the implementa- 
tion of the evaluand. That would be called context evaluation. The data 
collected at this stage of evaluation will describe the context in which 
the program ^will have to be implemented and will provide the base-line 
data for comparisons later on. 

Some implementation of the evaluation effort will be conducted 
during the conduct of the program, dealing with program formation, collect- 
ing information on inputs (input evaluation) and on the processes of 
intervention (process evaluation). 

Another important part of the implementation of evaluation will be 
contingent upon the implementation of the program, and will be the 
evaluation of impact . One cannot talk of impact evaluation or output 
evaluation unless there is a program on the ground; and unless the program 
has had some time to take effect and show results in the lives of people 
and communities. 
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Evaluatloa Planning of Post-Literacy Programs; 
The Indian "Scene 

In an optimistic and patriotic state of mind, one could find a lot 
to boast about literacy and post-literacy programs in India during the 
last 'few years. However, a more professional and sober analysis would 
show that while a lot has been done, a lot more remains undone. A variety 
of innovative programs have been implemented over the years, among them, 
prize schemes for writers and publishers of books for new literates; 
writers' workshops and production of reading materials; and a variety of 
library schemes for taking reading materials to readers in, rural areas ^ 
and industrial slums. Some evaluation and research work has also been 
done in the areas of readers' interests, vocabulary studies, and retention 
of literacy. 

As part of this planning exercise, we do not seek to present: a 
profile of all the post-literacy programs that have been implemented in 
India or to review the various evaluation studies that have been conducted, 
published and disseminatesd . Here, we need to ask a futuristic question: 
What evaluation studies should be conducted in the context of post- 
literacy programs in India today? What might be a list of evaluation 
studies that are critical to the conduct of effective post-literacy pro- 
grams in India at this point in the history of literacy promotion in the 
country? This is not to suggest that we need not look back. We do need 
to evaluate our past experience and learn f roof the post-literacy programs 
that* we have implemented for years,*...jpme of them for decades. We 

do need to evaluate to consolidate. 
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Infomatioiv-for Decision-Making; 
Evaluation^ Research, Appraisals . Monitoring 

So far in our " discussion we have talked only of "evaluation" of 
post-literacy programs. We have use'd-the term evaluation generically .to 
cover the generation and collection of all information needed for 
management and decision-making. 

Jn the literature of evaluation, the word evaluation has come to 
acquire special. meanings, and is distinguished from research, appraisals, 
monitoring and mahagement information system design. 

Evaluation, for instance, is now defined as the process of generating 
cont^ext and feedback information - systematically, objectively and with 
exactitude - with the objective of improving the conduct of an ongoing 
program and serving related d4cision-making needs. 

Research, even though it uses the same. tools and technology as 
evaluat^i.n, differs from it in terms of its objectives. The objectives 
of evaluation are to produce knowledge for use within the contex^ of the 
program within which the evaluation question arose. T.e evaluator's 
primary' loyalty is to the program; everything else is incidental. The 
objectives of research are to create general izable knowledge; to add to 
the common fund of human kno?,]^dge, typically on the basis of empirical 
data . 

Appraisals (or quick appraisals) are the children of necessity. 
These are evaluations conducted in a short period of A to 6 weeks on an 
emergency basis to examine the strengths and weaknesses - of a program and 
to identify the factors responsible for the prevailing conditions. 

Monitoring is to check upon the ongoing program for flaws and break- 
downs to enable the decision makers to regulate program activities and 
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to undertake corrective actions. Monitoring activities do not involve 
the study of causes or relationships but typically ^ay with matching 
observation* with expectations and with the developinent of performance 

profiles, " ^ • f ' 

Management Information System (MIS) design involves the construction 
of a system for orgariizing routinely generated data within^a program, for 
better management of the program. At its best, an MIS would help in all 
'the three processes of quick appraisals, ^valuation^and research. 

The Weed for ''System Thinking" in Plannin g 

^In doing evaluation planning for post-literacy, as indeed in all 
planning, system th'inking is no^Just important, it is essential. System 
thinking, as opposed to linear thinking, is a habit -of mind that accommo- 
dates "atonceness*' and a multiplicity of related variables and parts in 
interdependence. Thus, a planner of post-literacy evaluation will have 
to consider post-literacy as a subsystem in a larger system that in 
turn JLncludes subsystems that produced literates and new literates and 
subsystems that provide opportun:.ties for employment , 'Apolitical partici- 
pation and social and cultural enjoyment. 

The system/subsystem relationships in the context of post-literacy 
may be presented as follows: 
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INTERFACE INFORMATION:. 
Information about the 
context of programs of 
literacy and post- 
literacy; about client 
groups;^ and about the 
performance of earlier 
programs of post- 
literacy . 
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1 Societal system — 

Figure. The relationships between and kmong the components of the 
subsystems of literacy and post- literacy , and between and among these 
two subsystems and the overall societal system. 
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The literacy subsystem as we can see from the diagram presented 
above is itself made up of three component programs. These program 
components, in turn, may be constituted by many different projects, 
both big and small. At tile other end, thf post-literacy subsystem is 
also composed of four program 4|>Dmponents. Hete again, each progrim 
component may be constituted by many smaller projects. These subsystems, 
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with all their component programs and ccnstitutent projects, are in 
interaction with the larger societal system — making it and being made hy 
it at the same time. 

There is another important sense in which the evaluation planner of 
post-literacy programs (as any evaluation planner for that matter) will, 
have to engage in system thinking. After having conceptualized the 
world of work in terms of systems, subsj^stems and component programs and 
constituent projects — second and third-order subsystems — the 
evaluation planner will have to think of each system, subsystem and 
their systemic parts in terms of the four i>a^;^eters of context, inputs, 
processes, and outputs. "Thinking with" these parameters, the evaluation 
planner will be able to ask questions about information needs at various 
points of the total complexity called 'tihe^ national post-literacy program. 

Evalujtbion Planning or Program Suggestions? 

Evaluation planning has to be done -in relation to the evaluand ~ 

» 

the program or projects which will be the object and focus of ^^a^luation. 
Some part of evaluation planning as was discussed earlier al\^s^, 
antici^ory-" it is done so that a useful program can be conceptualized, 
designed and implemented. In cases where a full-fledged program is not 
already on the ground, evaluation planning has to be mostly anticipatory. 
In our particular case, for instance, a lot of evaluation planning will 
become program planning. In suggesting what might be useful context 
evaluation arid what might be useful feedback information, we will be 
talking about what might be useful to do in the first place and what kinds 
of post-literacy programs should be established and tested-in-use. 

13 
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Evaluating Post-Literacy; 
Some Questions Worth Pursuing 

We now return to the diagram on page 10 showing tba relationships 
between the subsystems of literacy and post- literacy on the one hand 
and the relationships of these two subsystems with the larger social 
system on the other. Thinking with the four parameters of systems given 
to us by systems theory — context, inputs, process(=2s, and outputs — we 
will generate a list of possible and what we think are worthwhile questions. 

It is worth repeating here that the following lists of questions 
are not meant to be prescriptive. The lists will have to go through two 
revisions — a real-time technical revision and a political revision. As 
part of the real-time technical revision, those forking with post- 
literacy programs in India will have to reinvent these lists in the 
context of their program needs, established priorities and available 
resources. As part of the political revision, the lists will have to be 
negotiated through political processes, on the one hand with policy 
makers and planners, and, on the other hand, with the beneficiaries of 
programs who will often provide the needed data to the evaluator. 

I. Some Questions at the Interface 

(i) Is instructional time of 300-350 hours spread over 8-10 
months sufficient to make adults in literacy classes literate? What 
about functionality? What about awareness? 

(11) What is the level of literacy acquisition of dropouts and 
school leavers coming out of elementary schools? 

(ill) What is the age, sex, occupation, income and class composition 
of those declared literate through adult literacy programs? 

H 
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(iv) What is the new literate's view of the illiterate? What is 
the illiterate's view o? the new literate? 

(v) Is there a difference between the literate and the 
illiterate in regard to their knowledge of ''how the world works"? 

(vi) What is the experience in reaching the "poorest of the poor" 
and other disadvantaged groups during the literacy program which could 
now be used to reach them in the post-literacy stages? 

(vii) What are some typical profiles of aspirations of peoples — 
illiterate, new literate, young, old, male, female? 

II. Literacy Retention: Some Evaluation Questions 

Some of these questions may be better characterized as research 
questions but they all have practical significance in relation to the 
design and implementation of a Rost-literacy program; 

•1 The Phenomenon of Retention 

(i) How many of those declared literate typically lose their 
literacy skills and under what conditions? 

(ii) Who retains literacy? What level of literacy makes a person 
retention prone? What are the motivations of those who are able to 
retain literacy? What are the uses to which they put their skills? 
What is their work and life context? 

•2 Provision of Reading Materials 

(i) How have the Indian prize schemes dealing with the production 
of reading materials done? What is the natural history of a prize 
winning book? How do the sales of prize winning books compare with those 
that did not win prizes? In other words, what is the economics of prize 
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publications? 

(ii) Do prize winning publishers later publish more books for 
new literates? 

(i^i) What is the level of productivity of prize winning authors? 
Do they write more books for new literates? 

(iv) Do nationally known popular writers want to write for the 

new readers? Why? Why not? 

(v) Is writers workshop a useful format for training writers of 

post- literacy materials? 

(vi) Is writers workshop a useful format for actual production of 

materials for new readers? 

(v±i) How do different models of writers workshops in use in 
India and abroad compare with eaph other in terms of productivity and 
economy, efficiency and effectiveness? 

(viii) When can writers cooperatives succeed in the provision of 
post-literacy materials for new readers? 

(ix) What has been the experience with translation of books for 
new readers from one language into another? What has been the experi- 
ence with co-publishing? 

(x) What has been the experience with the adaptation of 
available materials (such as extension materials in agriculture, health, 
nutrition and family planning) to the special needs of the new literates 

and now literates? 

(xi) What has been the experience with readers-written books? 
Or. with the writing of books based on the oral history of a community? 



^hola H.S. 1981. Writing for p^^Hpv^; A Book on Follow- 
Books. Bonn, FRG: German Foundation for International Development. 
190 pages. 
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♦3 Vocabulary and Writing Research and Evaluation 

(i) What is the spoken vocabulary of the prospective participants 
in programs of post-literacy? 

(ii) What written words have become sight words for those coming 
out of literacy classes and for those coming out of schools? 

(iii) Can we taxonomize levels and contents of literacy in terms 
of specialized word lists? 

(iv) What words must be mastered by learners to cope with 
scientific agriculture, family planning information, membership in 
cooperatives and such? 

(v) What factors should entar into the design of readability 
formulas for measuring the readability levels of reading materials i:i 
Hindi and other Indian languages? 

(vi) What are the reading interests of adult men, adult women, 
and youth as expressed by them? What reading materials are actually 
chosen by them under conditions of free choice? 

(vii) How to do content planning of books for new readers to 
relate them to their information needs on the one hand and their existing 
world-views on the other hand? 

(viii) What i| the nature of good writing for new readers as 
discovered from the utilization of books already written for them? 

(ix) What are the special considerations in regard to type size, 
illustrations, book size and binding for the literature for new 
litarates? 

(x) What might be the special features of the so-called 
problem-oriented reading materials? 
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(xi) What might be the special features of materials written 
for reading aloud by a literate to a group of illiterates? 

(xii) What fonnat(s) might be selected for post-literacy and 
extension materials so that these are not seen as ephemeral but can be 
Stored and filed for later reference by reading circles? 

(xiii) How to go about field testing of reading materials? Can 
this process be routinized? 

•4 Delivery of Services and Organizatfional Aspects of 
Post-Literacy Programs Dealing with Retention 

(i) What are the comparative advantages and disadvantages of 
models for the delivery of post-literacy such as those proposed by the 
Naik Committee: village continuing education centers; mobile library 
and continuing education services; continuing education and adult 
education centers; need based continuing education (school certificate) 
courses; follow-up activities conducted by students; and diversification 
of existing village libraries? 

(ii) What are the advantages and disadvantages in the operation 
of supervised post-literacy programs (such as programs offered in 
organized groups with trained tutors) and independent post-literacy 
programs (where learners make independent decisions to borrow reading 
materials) • 

(iii) What are the profiles of good libraries? What are the 
problems of unsuccessful libraries? 

•5 Recruitment and Training of Cadres 

(i) \^at are the special qualifications and characteristics, 
and training needs of post- literacy workers? 
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(ii) Can literacy teachers become good organizers of post- 
literacy programs? 

(iii) Can teachers of post- literacy programs be mobilized 
through political work or do they have to be more formally recruited? 

"n 

•6 Habits of Media Consumption and Other Effects 

(i) What do people do with what they read? 

(ii) What are the reading habits of new literates and now 
literates in regard to the use of rural newspapers and special magazines 
written for chem? 

(iii) Does literacy and reinforcement of literacy lead to a new 
"technology of intellect"? When does the new way of classifying, 
reasoning and remembering take hold? 

(iv) Can reading be encouraged by serializing and clustering 
books and asking people to "Read a Hundred Books" or to "Read a Thousand 
Books'' as they have tried in China? 

(v) Do literates in post- literacy programs have habits of 
information consumption different from those who never enrolled in such 
programs? 

(vi) How does the utilization of radio and TV differ among groups 
of illiterates, new literates, and now literates enrolled in post- 
literacy programs? 

(vii) How does literacy and post- literacy effect the relationships 
of literates with traditional media such as attendance at and enjoyment 
of puppet shows and the like? 

(viii) Does literacy and post- literacy change the relationship 
between the new literate and the power figures in his or her environment, 



er|c 



19 



18 



such as, the village headman, policeman, extension workers, government 
officials, and, in the case of women, men in general? 

III. Some Evaluation Questions on 

Second Chance Education Programs 

Evaluation questions in this particular program stream of post- 
literacy can deal with the characteristics of clients of these programs, 

with curriculum, delivery of instruction, problems of legitimization of 

I 

second chance education and effects of second chance education on 
graduates and on competitive educational systems: 

(i) What is the demographic profile of those joining or wanting 
to join second chance education programs? What are their future 
aspirations? 

(ii) What are the special needs of participants of second chance 
education programs? What are the problems of creating programs that 
are different in content but equivalent in objectives at various levels 

of formal education? 

(iii) What are the comparative cost-effectiveness ratios for 
different modes of delivery for second chance education: evening 
schools, correspondence education, accelerated day school? 

f (iv) What is the legitimization given to graduates of second 
chance education programs within the social and economic system? 

(v) What are the effects of second chance education on 
participants in relation to personal satisfactions, self-concepts, 
educational mobility, vocational success and increments in income, 
social prestige and political participation, etc? 

(vi) What are the effects of second chance education programs on 
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competitive systems of formal education and on nhe dernocratization of 
education in general? 

IV. Some Evaluation Questions on the 
Systemic Integration Theme 

Some general questions can be asked about the very concept of 
integration in operation , in other words, on the operational meaning 
of integration; on the special kinds of cadres needed for effecting 
systemic integrations and the kinds of values needed among cadres for 
them to work towards such integrations. There is information needed on 
the economic and political opportunities available, on the problems of 
codification of information needed for more effective performance of 
institutions and the effects of integration on new literates and on 
existing institutions. Here are some examples of questions: 

(i) What do post-literacy workers do when they claim to work 
on the task of integrating new literates into-ihe social, economic and 
political structures of the society? ]\ » 

(i^i) What are the characteristics, qualifications and value 
profiles of those working in programs of systemic integration? Are they 
aware of being engaged in tasks which are basically reconstructionist 
and egalitarian? Are their values congruent with the tasks they have 
been recruited to perform? Can civil servants or apolitical cadres 
accomplish the tasks of systemic integration? 

(iii) What is the range of institutions ~ traditional and 
secular — in wliich there are opportunities for new literates to 
participate? 

(iv) What has been the success of polyvalent adult education 
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centers in urban areas in economic integration of their participants? 

(v) What are the possibilities of codification in writing of 
information and instruction for each available institution? In what way 
can written materials help in the capacitation of individuals for better 
role performance? How does the situation differ in regard to traditional 
versus modern institutions? Are some or most secular and modern 
institutions also working on the assumption of illiteracy among their 
members and clients? 

(vi) Do those who undergo post-literacy programs connect better 
with institutions in the society and demonstrate better integration? 

(vii) Do post-literacy programs seeking to promote systemic 
integration change the institutions themselves in regard to their 
leadership patterns and functional effectiveness and overall responsive- 
ness? Are disadvantaged groups participating in leadership and holding 
instructional roles within these institutions? 

(viii) Does post-literacy result in dysfunctional adaptation, 
making it impossible for new literates to continue to work in the 
informal economic sector in rural areas and forcing migration to the 
city to join the formal economic sector? 

(ix) - Is political awareness actually resulting from post-literacy 
programs? With what consequences for new literates themselves and for 
the advantaged classes? 

(x) What are the effects in terms of a general response to 
modernization of teaching scientific literacy to adults? 

(xi) What are the ef fee's on university students of their 
participation' in the National Service Scheme in regard to their political 
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resocializatioii? 

(xii) What kinds of programs of systemic integration invire 
greater^ community participation? 

V, Socialization for an Ideal Society 

* Many of the information needs 'f or this stream of post-literacy 
programs will hive been met through the questions already raised. A 
few more questions are added here: 

(i) Do post-literacy programs make independent learners or is 
there a contl'adiction in establishing supervised groups and in offering 
organized programs to adults whom we want to become independent learners? 

(ii) Can one identify a general propensity or inclination among 
adults in post-literacy programs or graduates of such programs to engage 
in individual and social praxis? 

(iii) Is there, among the relevant groups of adults, evidence of 
media appreciation, of understanding technology, of ecological and 
environmental concerns and of the need for a new international order 
based on peace and brotherhood of man? 

(iv) Have participants in post- literacy programs become better 
consumers of information and culture? 

(v) What are the possibilities of establishing "enabling" 
institutions that -will promote socialization of individuals for an ideal 

society? ^ 

(vi) What happens to those who conduct post-literacy programs 
in the field as instructors? Do they join the new secular leadership 
in rural and urban communities? 
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The preceding list of questions represents the theoretically 
possible according to just one individual specialist. (Hopefully a 
good one!) Another literacy expert could come up with a somewhat 
different list, though overlap between any two such listf: will be 

impossible to avoid. 

What is theoretically possible is not what evaluation specialists 
will want to do in full. They will have to make short lists that seem 
to meet their priority needs, within the real life and time of their 
programs? And that- is not all. The lists drawn by technical profes- 
sionals may (if not always will) have to be approved, adapted and 
sometimes vetoed by poiit:^cal actors involved in programs or concerned 
about their consequences . 

Evaluators as technicians have to be convinced that the task of 
developing these lists is not useless simply because political actors 
will be involved and will not gratefully accept all their suggestions 
so carefully crafted. Evaluation planning is not all logic, but it 
need not be all politics either. By developing good evaluation agendas 
that meet the criteria^of need, practical feasibility and economy, 
evaluators can make logic a part pf the politics of evaluation; they 
can assist political actors in making sensible decisions and make it 
politically expensive for them to be merely political. 

A Management Information System (MIS) 
for Post-Literacy Programs 

In describing the needs for information in regard to tV^ context 
of programs, their formation and implementation and feedback on their 
performance, we have been using the term evaluation in a generic sense. 
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It should be clear, of course, that some of the answers to the questions 
listed above may be better answered by research than by evaluation (see 
the distinctions made in the earlier part, of the paper). Also some 
of the evaluations may be conducted as quick appraisals and through 
monitoring of the programs by actual visits^ or on the basis of periodical 
reports from the field. 

All of the processes of research, evaluation, quick appraisals 
and monitoring can be helped by collecting and systematically organizing 
information routinely generated by and within the program. Such informa- 
tion can first be used in making every day management decisions and 
then in evaluating effectiveness of decisions, actions and their conse- 
quences. Such a 'system of information is called a management information 
system (MIS) • 

* It is not within the scope of this paper to suggest the outlines 
of such an MIS. All we seek to do here is to suggest that the 
possibility of establishing such an MIS should be seriously considered; 
and that such an MIS must be built upon the MIS established for the 
"parent** literacy program which is producing most of the new literates 
now being served by the various pot>t- literacy programs. 

Models and Methods of Evaluation 
in Post-Literacy Programs 

In the choice of models and methdds of evaluation of post-literacy 
programs, one must learn to be eclectic. One must use what we have 
elsewhere described as the 3-S (Situation-Specific Strategy) model. 
The evaluator must begin with the problem and then use o^'adapt the 
model or models that will help in the systematic handling and solution 
of the problem. 
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In a similar vein, one must chose situation-specific methodologies 
for the collection of data. In some cases, classical and quantitative 
approaches to data collection may be appropriate • In another setting, 
the information may be best collected .through use of naturalistic and 
qualitative methodologies. In yet another case, evaluative information 
may be available through the analysis of the discourse of those involved 
in the program as teachers or as learners* 



Administration of. Evaluation and 
Dissemination of Evaluation ResulC:s 



Evaluation planning is not simp3.y a matter of coming up with a 
list of evaluation questions. The evaluation planning process must 
include the problems of administration of the implementation process; 
and, more importantly, the planning for the utilization of the evaluative 
information produced. 

1. Administration of Evaluation 

The tasks in the administration of evaluation include the develop- 
ment of an evaluation system; linking this evaluation system with the 

V r 

program syllem as well as with outside support institutions; recruitment 
and training of personnel to conduct the evaluation; resource generation 
and allocation; provision of logistical support, quality control and 
time budgeting of the enterprise; and ensuring appropriate and ethical 
use of information collected, 

if 

(i) Developing an Evaluation System ^ 

To make evaluation possible, an evaluation system must be 
developed. This will typically involve the creation of a subculture of 
'evaluation — many different people, playing many different roles at 
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/ various levels of the program system* Some will be holding exclusively 

evaluation 'jobs, that is, they will not be engaged in program imple- 
mentation. Some will be program people, generating and colle<rring^\ 
evaluation data and feeding it into the e^luation system^^Iij some 
cases, evaluation information will be collected and supplie^-^by^ 
people, and institutions outside^the boundaries of post- literacy 
programs. The essential point is that th^re wlj^ be a mix of full-time 
and part-time evaluation roles within all evaluation systems; and that 
there must be a clear understanding about the distribution of labor 
within these systems. 

The above remarks apply only to a system of internal evaluation. 
Some evaluation may be ordered from external evaluators. In such a case, 
^ those responsible for the administration of evaluation must establish 

modes of interaction, and information sharing between the internal and 
external evaluation teams. 

(ii) Recruitment and Training of Evaluators 

All those recruited to full-time roles of evaluators within 
an^evaluation system, and those given part-time collaborative tasks 
within the evaluation system, must be properly trained and/or^ provided 
suitable orientation to the objectives, plans and procedures of the 
overall evaluation effort. All participants in the system must under- 
stand clearly what their obligations are, as also what are the rewards 
and credits due to them for their interest and labor. Sharing the credit 
for evaluation completed is a very important consideration, for no one 
wanes to put in hours and hours of labor so that somebody else can write 
a. report and advance his or her own career. 
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(iii) Linkage with Support Institutions 

Linkages with support institutions are important. 

Evaluators of post-literacy programs, employed directly within projects 

and programs, may never be able to carry the whole burden of research 

and evaluation by themselves. They will need the help of universities 

to do the needed research and evaluation. They will need also the 

IP 

assistance of publishers, libraries and voluntary agencies. 

In the case of India, it seems obvious that evaluation tasks will 
have to be divided between and among the Directorate of Education; the 
departments of adult and nonforraal education in the States; the 
infrastructure of State resource centers "nd some of the district 
resource centers; other departments dealing with development extension; 
and libraries, publishers and voluntary agencies, 
(iv) Resources, Logistics, and Control 

Evaluation requires resources and these are not easy to 
obtain when they compete with program needs within an overall condition 
of resource scarcity. Evaluation planners must learn to do most with 
the least of resources. This would mean that evaltiators must make 
evaluation part of programing. .Evaluation should be seen as routine 
as what one must do to implement a program effectively. Such a stance 
will allow the evaluator to piggyback on the resources already available 
to programs. 

In the Third World settings, where evaluation of post-literacy 
programs will often take place, logistics of sample selection, data 
collection and data analysis are going to present serious difficulties. 
Evaluators can not play God and will that there be telephones, and that 
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there be roads and that there be all the cars and landrovers needed for 
their trips upcountry. All they can do is plan in a way that logistical 
problems are anticipated and as far as possible are accounted for. 

Finally, there is the problem of control: to ensure that the 
tasks get done; that the accomplishments is of high quality; that data 
are not fabricated or falsified; that tasks are completed on time; and 
that those participating in the process of evaluation can keep a high 
morale — consider the work worth doing and can see its usefulness in 
the actual improvement of prograios . 

2. Synthesis of Evaluative Information and 
. Dissemination for Utilization 

Evaluation studies are of no use, if information collected is not 
used in improving program or project performance. In the case of a 
large program, with a large evaluation system, with many participants 
at various levels and in various institutional settings, it is absolutely 
necessary that due attention is paid to the need for collecting the 
various evaluation studies conducted within the program; for synthesizing 
the findings of these various studies; drawing practical, operationa''. 
conclusions from the synthesis of rasults; and for feeding these ideas 
back to all concerned. 

As a general principle, those who collect data must be encouraged 
to use it before feeding it into the system. Indeed, the forms and 
profiles for data collection can be so designed that the consolidation 
and display of data itself compels insights into the performance of the 
program. Syntheses of evaluative information should be done at various 
levels of the system as drta moved towards a central point. At the 
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central point, a final synthesis should be made; and the practical 
implications of findings disseminated throughout the system — among 
evaluators and program specialists. 

The synthesis and dissemination will not happen unless it is 
planned for and unless suitable knowledge linkage roles are created. 

ConcludinR Remarks 

An evaluation planner, in a mischievious mood might write me some 
day to say: I have read your paper, followed your suggestions, and come, 
up with an evaluation agenda which,! always knew I should have! The 
enterprise of writing such papers, and reading and using them in our 
work is not as useless; however, as such a smart remark might suggest 
For there is indeed a difference in the intuitive knowing and th4 
. systematic knowing, even if the content of knowing were almost the s^e. 
By following the process we suggest, we will understand the program ^ 
better and we will be able to come up with justifications for choic/s 
that can satisfy both ourselves and others. 
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